23 June at 11:00 AM
Ferrari 1 Building, Via Sommarive 5, Povo (Trento)
Room n. 259, Augmented Health Environments Lab

Speaker: Dr. Firoj Alam (firojalam.one)

Abstract

Robust benchmarking is essential for understanding and advancing the capabilities of large language models (LLMs) across diverse linguistic and cultural landscapes. This talk explores the critical role of benchmarks in evaluating LLMs’ proficiency in native, local, and cultural knowledge, with a particular emphasis on multilingual and dialectal challenges.

This talk will present recent research on developing frameworks and resources for assessing the capabilities of LLMs in multilingual and dialectal contexts, with particular emphasis on native, local, and culturally-aligned natural queries, as well as everyday spoken queries. These new benchmarks and resources offer a comprehensive perspective on the strengths and limitations of current LLMs, highlighting opportunities to advance more inclusive and culturally aware language technologies. Additionally, the talk will present the importance of domain specialization in LLMs to enhance their effectiveness on domain-specific tasks.

Bio

Dr. Firoj Alam (firojalam.one) is a Senior Scientist at the Qatar Computing Research Institute (QCRI), Qatar. With over a decade of experience in AI, NLP, and speech technology, Dr. Alam leads multiple large-scale research projects on large language model benchmarking, native and cultural alignment, generative AI content detection, and disinformation analysis. He serves as Lead Principal Investigator for several funded initiatives, including projects on native and cultural inclusivity in LLMs, media bias and factuality profiling. Dr. Alam has authored nearly 100 peer-reviewed publications and contributed significant resources and tools to the research community. An active member of the global AI community, he regularly serves on the program committees of top-tier conferences such as ACL, EMNLP, and NeurIPS. Dr. Alam is also an alumnus of the University of Trento.